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"Method and system for coding an image sequence, corresponding coded signal and storage 
medium, and method and system for decoding such a coded signal". 



The present invention relates to a method of coding an image sequence, 
comprising the steps of : 

transforming the images of said sequence into successive segmented 
images comprising a plurality of regions and associated labels defining partitions ; 
5 - estimating for the regions of the current partition a motion information 

about the evolution of said segmentation from the previous partition ; 

encoding the contour and texture of each region of said current partition ; 

generating on the basis of said encoded partition a corresponding 
reconstructed partition. 

10 

The invention also relates to a system for coding an image sequence, 
comprising : a segmentation device, for generating successive partitions comprising a 
plurality of regions, a motion estimation device, for the evaluation of the motion of said 
15 segmentation from a previous partition to the current one, an order information evaluation 
device, for defining a priority in the coding order of said regions, a coding device, for 
coding contour and texture signals for all regions of the current partition, and a partition 
reconstruction device, and to a coded signal as obtained at the output of such a system and to 
a storage medium for storing said coded signal. 

20 

The invention also relates to a method of decoding coded signals 
corresponding to the regions of each successive partition of a segmented image sequence, and 
to a decoding system for carrying out said decoding method. 

In the following description, the word "image* 1 has to be understood 
25 according to its widest meaning. It is indeed clear that the invention deals not only with 
conventional images in which at least a luminance signal is associated to each picture 
element, but also more generally to any set of tridimensional data (u, v, w) among which 
two of them define a kind of position within a predetermined frame and the third one a value 
varying within a given range and respectively associated to each pair of data. The proposed 
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method offers however a substantial interest for all segmentation-based coding techniques and 
can be used particularly in video terminals that use a motion compensated coding of the 
partition issued from the segmentation. Such techniques are mainly useful for very low bit 
rate applications, and a standardization under the reference MPEG4 is contemplated (MPEG 

5 is an acronym for "Moving Picture Experts Group", which is a group of experts of the 
International Standardization Organization ISO). 

The basic idea of segmentation-based coding of image sequences is to 
define a partition of said images that is more appropriate for coding than the conventional 
block-based coding schemes. A conventional segmentation-based coding method involves 

10 mainly the following steps : definition of the partition, coding (and transmission) of the 
partition, coding (and transmission) of the pixel values inside each region defined by the 
partition. Two modes of transmission can be distinguished : the intra-frame mode, which 
consists in sending the information about the partition of each frame independently, and the 
inter-frame mode, which relies on the characterization and coding of the time evolution of 

15 the partition from one frame to the next one (and is often much more efficient than the 
intra-frame mode). 

The inter-frame mode of partition coding generally relies on the following steps : 



(a) segmentation of the frames (the present invention will address the general 
problem of partition coding by motion compensation and does not rely on any specific 

20 segmentation scheme) ; 

(b) estimation of the motion of the partition between an earlier partition and 
the current one ; 

(c) prediction by motion compensation of said earlier partition ; 

(d) computation of the prediction error ; 

25 (e) coding of this prediction error, the coded error being then transmitted 



(individually for each region of the partition). 

The motion estimation analyzes the movement of objects between two 
successive frames and leads to a motion information represented by a set of parameters that 
are assigned to each region. This motion information, combined with a stored reconstructed 
30 partition, allows to carry out the motion compensation, and the difference between the 

current segmented frame and the reconstructed partition gives the prediction error, which can 
be coded and transmitted. 

However a distinction has to be made between the motion of the shape of 
a region and the motion of the pixels inside a region. Indeed, both notions coincide in the 



BNSDOCID: <WO 970S746A2_I_> 



WO 97/05746 PCT/IB96/00747 

3 

case of an object placed in the foreground of another one : either the contours of the region 
of the partition corresponding to that object or its texture can indifferently be considered, the 
pixels of the interior of this region and its contours following the same motion. This is no 
longer the case for a background object, because the modifications of the shape or contours 
5 of the region corresponding to this background object are generally not defined by the motion 
of its interior but by the motion of the foreground regions overlapping that background 
region. 

For background regions it is therefore necessary to perform two different 
motion estimations, in order to have different motion compensations for contours and for 
10 textures, the first estimation being used for contour coding and the second one for texture 
coding. 

The communication "Motion and region overlapping estimation for 
segmentation-based video coding" of M. Pardas and P. Salembier, IEEE International 
Conference on Image Processing, Austin, Texas (USA), November 1994, volume II, 

15 pp.428-432, describes an object-based video coding system in which it is indeed avoided to 
perform two different motion estimations for the background regions. The motion estimation 
for contours is replaced by an extra information called order relation between regions and 
which allows to know for every couple of neighbouring regions which of them is in the 
foreground. To. this end one performs for every boundary between two neighbouring regions 

20 a two hypothesis test consisting in calculating the prediction error in each region for the two 
possible order relations and selecting the order that yields the lowest one. It however appears 
that the implementation of this method of determination of said order information leads to a 
very high computational load : a prediction error has to be calculated twice for each region 
and, moreover, a lot of these computations are not useful since the corresponding regions 

25 will finally perhaps be coded according to the intra-frame mode, i.e. without re-using the 
error predictions. 



It is therefore a first object of the invention to propose a different and 
30 more efficient coding method, leading to a reduced and less expensive computational load. 
This object is achieved by means of a method as defined in the pr nble of the description 
and in which the motion information estimation step comprises in cascade the sub-steps of : 

defining for each region of said current partition, on the basis of a 
comparison between this current partition and a partition compensated from said 
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reconstructed partition and said motion information, an intra-frame or interframe contour 
transmission mode ; 

defining for each pair of neighbouring regions of said current partition, on 
the basis of a comparison between respective motion informations associated to these two 

5 regions, an auxiliary order information and an updated motion information. 1 
The proposed method is more efficient than the one described in the 
previously cited document in that the determination of the order information is carried out by 
way of a global estimation of both the mode of transmission and the order information, said 
mode of transmission being taken into account for each region of the partition issued from 

10 the segmentation and no longer by considering that all regions of a given frame are 
transmitted in the same mode. 

In a more efficient implementation, the method includes a contour 
transmission mode definition sub-step in which information is sent backwards in order to 
constitute a loop in which said order information updates the definition of said contour 

15 transmission mode. 

According to the invention, this method may also be advantageously 
characterized in that : 

(A) said contour transmission mode definition sub-step comprises for each 
region individually the operations of : 

20 - compensating on the basis of the motion information associated with each 

successive region of the current partition said corresponding reconstructed partition ; 

computing on the basis of a comparison between said current partition and 
said corresponding compensated partition a compensation error ; 

generating as a motion information and on the basis of a comparison 
25 between coding costs of said current partition and said compensation error a decision related 
to the transmission mode of the contour of the concerned region ; 

(B) said auxiliary order information definition sub-step comprises the 
operations of : 

defining a motion vector for each position inside the concerned region and 
30 assigning said vector to all points of the current partition ; 

generating the order information corresponding to each of the concerned 
regions, on the basis of a comparison between said current partition and said reconstructed 
partition when considering said assigned vector ; 

quantizing said order information by comparing for each pair of labels 
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associated with two concerned regions the number of occurrences where the first label has 
been declared to be in the foreground of the second one and the reciprocal number and 
selecting the largest one. 

A further object of the invention is to propose a coding system for 
5 implementing a method as previously indicated. 

This object is achieved by means of a coding system as described in the 
preamble of the description and in which said order information evaluation device is a 
decision-directed sub-system comprising a transmission mode definition circuit, provided for 
assessing the coding costs of the contour information in intra-frame and inter-frame modes 
10 and defining a first updated motion output signal including an updated motion information 
and a corresponding transmission mode information, and an order computation circuit, 
provided for associating to said first updated motion signal a second order signal and 
comprising : 

- motion field expansion block, in the form of a first scanning loop for 
15 extending, for each region, its motion model to all points of the partition and defining an 

order information for said concerned region, and of a second scanning loop for iterating said 
motion field expansion for all regions of each current partition ; 

- an updating block ; 

- a quantization block, for taking a majority rule decision related to the 
20 final order of each region ; 

the output of said order information evaluation device associating said final order signal and 
said updated motion signal. 

A further object of the invention is to define a coded signal such as 
generated by such a coding system, said coded signal corresponding to each region of each 

25 successive partition of a segmented image sequence and comprising, for each pair of regions 
of a partition considered as in conflict as soon as their contour and texture motions differ, a 
first coded signal, corresponding to the texture signal of each concerned region and to the 
corresponding motion information that characterizes the evolution of said region between the 
previous partition and the current one, and auxiliary signals, corresponding to an order 

30 signal, defining for each considered pair of regions said in conflict which of them is in the 
foreground of the other one, and to an associated updated motion information. A further 
object of the invention is to propose a storage medium for storing said coded signal. 

A further object of the invention is finally to propose a method of 
decoding such a coded signal, said decoding method then comprising the steps of : 
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(a) decoding the regions that have been previously coded in intra-frame mode 
and creating a corresponding mask by assigning the corresponding labels to these regions and 
a zero label to the other ones ; 

(b) generating a compensated partition made of intra-frame coded regions, 
5 motion compensated regions and holes corresponding to empty areas ; 

(c) decoding the error regions and generating a socalled over-partition made 
of the intersection between the compensated regions and the set of transmitted error regions, 
the labels of the regions of said over-partition being decoded and assigned ; and to propose a 
corresponding decoding system. 

10 



These and other aspects of the invention will be apparent from and 
elucidated with reference to the embodiments described hereinafter and to the drawings, in 
which : 

15 Fig.l is an illustration of the method of segmentation according to the 

invention. 

Fig.2 illustrates, for the motion estimation of the regions of the current 
partition, the vectors that, for each represented region, describe the time evolution of the 
region in a backward mode ; 
20 Fig. 3 illustrates the partition compensation provided for defining a 

prediction of the current partition ; 

Fig. 4 illustrates a forward motion compensation of partition, and Fig. 5 a 
backward motion compensation of partition ; 

Figs 6 and 7 illustrate the shape coding in intraframe mode and in 
25 inter-frame mode respectively ; 

Fig. 8 shows an embodiment of a coding system carrying out the method 
according to the invention, and Figs. 9 to 1 1 show in a more detailed manner some devices 
of the coding system of Fig. 8 ; 

Fig. 12 is an illustration of a method for decoding signals coded as 
30 indicated above, and Fig. 13 shows an embodiment of a decoding circuit included in a 
decoding subsystem carrying out this method. 

The invention will now be described while using the following notations : 

- the original images at times (T-l), T, are called ORIG(-l), 
ORIG(O), and so on ; 
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- the partitions are called SEG(O) for the current partition at time T, 
SEG(-l) for the previous partition at time (T-l), and so on if necessary for other partitions at 
other partitions at other times; 

similarly REC(O) and REC(-l) designate the current (at time T) and 
5 previous (at time (T-l)) partitions as they would be reconstructed on the receiver side after 
transmission, and as they are obtained in the encoding device (REC(O), REC(l), are not 
exactly the same as SEG(O), SEG(l), owing to the losses during the partition coding 
process) ; 

MOT(O) is the motion information that characterizes the evolution of the 

10 partition between (T-l) and T (that is to say that allows to deduce SEG(O) from SEG(-l)), 
and is generally represented by a set of parameters that are assigned to each region of the 
previous partition SEG(-l). 

As already said, the objective of the invention is not to address the 
problems of segmentation and motion estimation, and it is therefore assumed that SEG(-l), 

15 SEG(O), REC(-l), MOT(O) are known. An implementation of a segmentation method is for 
instance described in the communication "Time Recursive Segmentation of Image Sequences'* 
of M. Pardas and P. Salembier, EUSIPCO 94, Vllth European Signal Processing 
Conference, Edinburgh (United Kingdom), September 13th, 1994, while for the purpose of 
motion estimation a method as described in the document "Region-based motion analysis for 

20 video coding at low bitrates", H. Sanson, Proceedings of MPEG4 Seminar, Paris (France), 
March 1994, can be proposed. 

These indications being given, the coding method according to the 
invention is schematically shown in Fig. 1. Said method comprises in cascade the following 
steps : first a definition of the transmission mode of each region (from la to 3c), and then the 

25 order computation (from 4a to 5). Indeed, in practice, it may not be always efficient to send 
all regions in inter-frame mode. For instance, if the motion estimation has not produced 
reliable results or if the motion of the region cannot be modeled by the motion estimation, 
the prediction error of the region shape may be so large that it may be less expensive to code 
directly the entire region (by switching from inter-frame mode to intra-frame mode). 

30 For the first step of definition of the transmission mode, the input data are 

the current segmented partition SEG(O) (obtained from the corresponding original image 
ORIG(O)), the previous reconstructed partition REC(-l), and the motion information 
MOT(O) associated to the partition SEG(-l). The objective of this first step is to define the 
transmission mode (intra-frame mode or inter-frame mode) of each region, by sending a 



BNSDOCtD: <WO 9705746A2_I_> 



WO 97/05746 PCT/IB96/00747 

8 

binary information used to say to the receiver whether or not the region is transmitted in 
intraframe mode for contour coding and stored for instance within the motion information 
MOT(O). This motion information MOT(O) will therefore involve a set of motion 
parameters for each region (as already said) plus a binary information indicating the 

5 transmission mode of the associated contours. The decision about this transmission mode is 
taken on the basis of the cost of the contour, by comparing the cost of each region according 
to the fact that it is sent in intra-frame mode or in interframe mode. As shown in Fig. 1, 
three sub-steps are considered in this first step. 

The first sub-step (la, lb) is provided for compensating the whole 

10 partition. Based on the previous reconstructed partition REC(-l), the motion information 
MOT(O) and the order information, called ORD(O), this partition compensation, comprising 
two operations (la) and (lb), will lead to a compensated partition called RECCOMP(O). 
Although these two operations are described later in a more detailed manner, the general 
philosophy of this partition compensation may be however discussed now, and is illustrated 

15 in Figs. 2 and 3. Before the compensation, the motion of the various regions of the current 
partition SEG(O) will be estimated. The obtained motion estimation gives a set of parameters 
describing the time evolution by each region in a backward mode BM. In Fig. 2, only one 
vector (MOT1 and MOT2) has been represented for each region of SEG(O), but the 
estimation generally gives more complex models defining one motion vector for each pixel of 

20 each region. Having noted that the motion of the pixels inside a region (texture motion) and 
the motion of the shape of a region may not be equivalent (both motions coincide in the case 
of foreground regions, for which the pixels of interior and contours follow the same motion, 
but it is not the case for a background region because the modifications of its shape or of its 
contours are defined by the motion of the regions in its foreground), it is assumed that the 

25 texture motion is used to compensate both the partition and the texture. The partition 
compensation problem is then shown in Fig. 3 : based on the previously reconstructed 
partition REC(-l) and on the transmitted motion parameters, the compensation should define 
a prediction of the current partition REC(O). 

The compensation itself can work either in a forward mode or in a 

30 backward mode. As shown in Fig.4, in the forward mode, the pixels of REC(-l) are 
projected towards REC(O). This projection can be done if the motion vectors MOT1 and 
MOT2, as defined by the estimation done in a backward mode, are inverted (MOT1 1 and 
MOT2 1). As can be seen, two problems may result from the transformation of the regions. 
Some pixels of REC(O) may have no label, they constitute the so-called empty areas (only 
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one, called EA in Fig. 4). By contrast, some pixels may have several label candidates, these 
conflicting areas are called overlapping areas (only one, called OA in Fig. 4). To solve the 
conflicts, the order information is used to decide which region is considered to be in the 
foreground of which region (in case of conflicts between labels, the foreground region gives 

5 the correct label). The problem of overlapping areas is specially important if the texture 
motion is used to compensate the partition because of the issue of foreground/background 
relationship between regions commented above. However, the use of the texture motion and 
of the order is a quite efficient solution because the texture motion information leads to a 
good compensation of the texture and the order only represents a small amount of 

10 information. Finally, the empty areas are left without label and are processed as 
compensation errors. 

The dual mode of compensation is illustrated in Fig. 5. It is a backward 
mode in the sense that, for each pixel of REC(O), one tries to look on REC(-l), which label 
has to be selected. In this case, the main problem is to define which motion vector has to be 

15 used when a pixel (i,j) of REC(O) is considered. Indeed, since the current partition is not yet 
known on the receiver side, one does not know the region the pixel belongs to and therefore 
its corresponding motion model. The solution consists in considering all possible vectors 
defined by all possible regions. In the case of Fig. 5, there are two regions, and two vectors 
are therefore considered for each point : one given by region 1 (MOT1) and one given. by 

20 region 2 (MOT2). Each time a vector as defined by a region n does not point to a pixel 
belonging to region n in REC(-l), the compensation is considered as being invalid and the 
corresponding vector (INV) is discarded (in Fig. 5, this is the case for two vectors). 
However, as in the case of forward motion compensation, some pixels have no valid 
compensation (empty area EA) and some others have more than one candidate (overlapping 

25 area OA). As previously, the order information is used to solve the conflicting areas. 

The main difference between the forward and backward modes of 
compensation deals with the quantization of the pixel locations. Indeed, generally, the motion 
vectors start from an integer pixel location but point to a non-integer location. In the forward 
case, it is the locations of pixels of REC(l) that have to be quantized whereas in the 

30 backward case, the locations of pixels of REC(-l) have to be quantized. There are some 
more difficulties related to the forward mode in the case of motion models involving 
modifications of the scale (zoom in particular). Indeed, in the case of region expansion, the 
modification of the distance between two pixels may create more empty areas in the 
compensated frame. These problems can be solved but generally the backward mode is more 
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simple. 

As previously indicated, the partition leads to a compensated partition 
RECCOMP(O). In a second sub-step (2), a compensation error ERR(O) is then computed by 
way of a subtraction between SEG(O) and RECCOMP (O). Finally, a third sub-step (3a, 3b, 

5 3c) is provided for estimating and comparing the coding costs of each region individually, as 
defined by SEG(O) (intra-frame mode) or by ERR(O) (inter-frame mode). The estimations 
(3a) in intra-frame mode and inter-frame mode can be done either by actually coding the 
information and measuring the resulting amount of information or by using any other 
technique that provides an approximation of the cost (for example, by assuming that the cost 

10 is proportional to the length of the contour of the region). Once both estimations have been 
performed, the comparison and selection (3b) can be done and the resulting decision is stored 
(3c) as the additional binary information updating the motion information MOT(0). 

The objective of this estimation is to assess the coding cost of the shape 
information if the region is coded in intra-frame mode or in inter-frame mode. The problem 

15 is illustrated in Figs 6 and 7. 

According to the first option (Fig. 6), it is considered that the region is 
sent in intra-frame mode. In this case, all the contours of the shape SH have to be coded. 
One can either actually code it by classical contour coding techniques and compute the 
resulting number of bits or make an estimation saying for example that the resulting cost is 

20 proportional to the length Lq of the contour of the concerned region (the cost can be 
estimated as C intra = KLq). The second option (Fig. 7) consists in coding only the 
compensation error. In the example of Fig. 7, only two small contour segments CI and C2 
have to be coded because the shape compensation corresponds quite well to the real shape. 
To get the cost, one can actually code the two segments of lengths Lj and L2 by classical 

25 techniques as before or make an estimation (for example C inter = 1,7 (LI + L2). The 
multiplicative factor is higher than previously to take into account the increased number of 
starting points. However, this factor depends on the technique that is going to be really used. 
Once both costs have been computed or estimated, the decision (selection of the lower cost) 
is taken by comparing them. If C inter is lower than C inlra , the region will be sent in 

30 inter-frame mode, otherwise it will be sent in intra-frame mode. The resulting decision is 
stored (3c) as the additional binary information updating the motion information MOT(O). 

The first step of definition of the transmission mode is followed by the 
second step of order computation, which can be decomposed, as shown in Fig. 1, in a first 
sub-step (4a, 4b) and a second sub-step (5). This first sub-step is provided for estimating a 
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set of order parameters and the second substep for achieving a quantization of the order 
information for the transmission. The first sub-step (4a, 4b) relies on a double loop, one of 
them scanning the regions defined by their label and the other one scanning the positions (i j) 
of the image space, and involves two operations in cascade. 
5 The first operation (4a) is provided for defining a motion vector for each 

position (ij) of the region corresponding to the concerned label. The motion information 
MOT(O) gives indeed for each region a motion model which is a function of the position 
(ij). For instance, if the motion model for a given region is a translation (dx, dy), then this 
motion vector is assigned to all points (i j) of the image. If the motion model is an affine 

10 model (al, a2, a3, a4, a5, a6), then the motion field at the point (ij) is defined as 

Vx (i j) = aji + aj + a 3 and Vy(i j) = a 4 i + a^ + a$. If the region has to be transmitted 
in intra-frame mode, this motion field expansion is not performed. 

The second operation (4b) is provided for updating the order 
corresponding to the concerned region. The order information consists in a list of possible 

15 regions in conflict, i.e. of regions of REC(-l) that may overlap with other regions during the 
motion compensation. Such an order list is set to zero at the beginning of the process. If one 
denotes by LABSEG(O) and LABREC(-l) the region labels of the current pixel (i j) in the 
current partition SEG(O) and of the pixel (i-Vx(i j), j-Vy (i j)) in the previous reconstructed 
partition REC(-l), the list entry corresponding to the conflict between the region of label 

20 REGNUM (REGNUM is a working label associated to each region successively considered 
in the current partition SEG(O)) and the region having the current label LABSEG(O) is 
updated if the following conditions are fulfilled : 

(a) the current label LABSEG(O) is different of any label of a region 
transmitted in intra-frame mode : in that situation (i.e. if the point (i j) in the current 

25 partition SEG(O) corresponds to a region which is transmitted in intraframe mode), any 
compensated label that falls in such a position is discarded ; 

(b) the previous label LABREC(-l) is identical to the label REGNUM : 
the point (i j) of the current partition SEG(O) is then a candidate to receive this label 
REGNUM ; 

30 (c) the current label LABSEG(O) is not identical to the label REGNUM : 

the correct label of the pixel (i j) is then not REGNUM, and the regions with the current 
label LABSEG(O) and the label REGNUM (=LABREC(-1)) are conflicting regions, the first 
one being in the foreground of the second one (the list entry corresponding to the conflict 
between the labels LABSEG(O) and REGNUM is then incremented by one unity). 
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This procedure is iterated for all points (i j) and for all labels REGNUM, 
the order list finally giving the number of occurrences where it has been possible to declare 
that a given region is in the foreground of another region. 

The second step (5) is then provided in cascade with the first one for 

5 quantizing the order (the receiver will only need a binary decision in order to be able to 
solve the situations with conflicting labels during the motion compensation). This 
quantization is achieved by comparing for each pair of labels the number of occurrences 
where the first one has been declared to be in the foreground of the second one with the 
number of occurrences where that second one has been declared to be in the foreground of 

10 the first one. The final order between the two concerned regions is defined as the one 
corresponding to the largest number of occurrences. This order ORD(O) may then be sent 
backwards for use as an input for the sub-step (la, lb), in order to constitute an iterative loop 
in which said order information allows to update the definition of the contour transmission 
mode. The number of the iterations is kept low if one wants not to increase the 

15 computational load. In practice, only two or three iterations are sufficient. It must be noted, 
in this case of an iterative loop, that the partition compensation cannot be done if the order 
information ORD(O) is not known. To solve this problem, it is simply supposed on a first 
iteration that all new regions (that is to say regions which are present in SEG(O) and not 
present in REC(-l) are transmitted in intra-frame mode and that all other regions are sent in 

20 inter-frame mode, which is equivalent to give to ORD(O) an initial value designated by the 
reference ORD 0 in Fig. 1. Finally, once the order information has been defined, the 
encoding step of the coding method, designed by the reference (6) in Fig. 1, can be carried 
out, for instance by following the contour/ texture coding approach described in the already 
cited document "Motion and region overlapping estimation...", IEEE-ICIP, Nov. 94. 

25 The above-mentioned partition compensation will now be described in a 

more detailed manner. This sub-step (la, lb) is intended to realize the compensation of the 
previous reconstructed partition REC(-l) and comprises several operations. Since the region 
a particular point (i,j) of the compensated partition belongs to is not known, all possible 
motion parameters assuming successively that this point belong to all possible regions (of 

30 label REGNUM) have to be tested. Therefore the first operation (la) is provided for defining 
a motion field for the entire image while using the motion parameters assigned to a single 
region, the current one of label REGNUM. For the operation (la), it is temporarily assumed 
that all pixels of the image belong to the region REGNUM. Generally this operation (la) can 
be easily done because the motion information gives for each region a motion model which 
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is function of the position (i,j). As for the operation (4a), if the motion model for a given 
region is a translation, this translation vector is assigned to all points (i,j) of the image. If the 
motion model is an affine one, then the motion field at every point (i,j) is defined by this 
model. This definition operation, similar to that one previously described (first operation (4a) 
5 of the first sub-step of the second step "order computation"), is followed by a second 

operation (lb) of compensation, according to which, for each pixel (i,j) of the compensated 
partition RECCOMP(-l), the compensated label is defined as follows (the procedure being 
iterated for all pixels (i,j) and for all regions, that is to say for all labels REGNUM) : 

(a) if the pixel (ij) corresponds to a region which has been transmitted in 
10 intra-frame mode, the label is not compensated : any compensated label that falls in that 

region is discarded ; 

(b) if the previous label LABREC(-l) (i.e. the region number of the pixel 
(i - Vx(i,j), j - Vy (ij)) in the previous partition REC(-l) is identical to the current label 
REGNUM), the compensation is valid and the pixel (i, j) is a candidate to receive the label 

15 REGNUM ; 

(c) if the location (i j) of the compensated partition is empty, the label 
LABREC(-l) must be assigned to that position ; 

(d) if the location (i,j) has already been assigned a label, this compensated 
label is in conflict with the previous label LABREC(-l), but this conflict can be solved by 

20 the order information stored in ORD(O) : the label is the one corresponding to the 
foreground region as defined by ORD(O). 

This procedure is iterated for all points (i,j) and for all possible regions of 
label REGNUM. At the end, the compensated information can be post-processed in order to 
create a partition. Indeed, the previous compensation technique does not guarantee the 

25 definition of connected regions and in practice several connected components may have the 
same label. To solve this problem, a two step procedure may be used. The first step selects 
one connected component for each label. Several criteria may be used. A simple one consists 
of selecting the largest component (note that this selection can be done at the receiver side 
without the need of transmission of any overhead information). The elimination of some 

30 connected components will create some holes in the partition, i.e. some regions which 
correspond neither to a region sent in intraframe mode nor to a compensated label. These 
holes can be left and processed as individual regions by the following step or they can be 
removed. For example, a hole can be eliminated by a propagation of the neighbouring labels 
or by assigning it to the largest neighbouring, etc. At the end of this procedure, one has the 
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final compensated partition RECCOMP(O). 

The coding method which has been described above can be implemented 
in several manners, for instance with the aid of wired electronic modules as described in the 
system of Fig.8, detailed in Figs.9 to 11. This coding system comprises in cascade a 

5 segmentation device 81 generating the successive partitions to be coded, a motion estimation 
device 82 for the evaluation of the motion of the regions of each current partition with 
respect to the previous one, an order evaluation device 83 for defining the priority between 
the regions of the current partition to be coded, a coding device 84 that implements the 
texture and contour coding step for each region of the current partition, and a partition 

10 reconstruction device 85. 

According to the invention, the order evaluation device 83 comprises a 
decision-directed sub-system organised in the form of a loop including a first circuit 91 for 
the definition of the transmission mode (intra or inter) of each region and a second circuit 92 
for the order computation. The first circuit 91 receives the current partition SEG(O) from the 

15 device 81, the previous reconstructed partition REC(-l) from the device 85, the motion 
information MOT(O) from the device 82, and the output order signal of the second circuit 
92. The second circuit 92 receives the output updated motion information available at the 
output of the first circuit 81, the current partition SEG(O), and the previous reconstructed 
partition REC(-l), and its output is the order signal sent both towards the fourth input of the 

20 first circuit 91 and, together with said updated motion information, towards the coding 
device 84. 

The first circuit 91, provided for comparing the coding cost of the contour 
information in intra-frame and inter- frame modes, comprises a selection sub-assembly (103 to 
106) for carrying out an estimation of the coding costs in intra-frame mode (in coding blocks 

25 103 and 104, at the outputs of which the resulting amounts of information are measured) 
and, after a comparison of both costs, the selection (block 105) of the smaller one and of the 
corresponding updated motion information (block 106). In the circuit 104, the coding cost is 
directly computed from the partition SEG(O), while in the circuit 103, it is computed after 
the previous computation (in a block 102) of a compensation error deduced from an 

30 estimation of a compensated partition. Said compensated partition is available at the output of 
a compensating block 101. However, as this compensation may not guarantee the definition 
of connected regions (several connected components may have the same label), a selection of 
only one connected components is done (for instance the greatest one, in a block 107) and 
the holes corresponding to the eliminated other ones (i.e. regions which correspond neither to 
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a region sent in intra-frame mode nor to a compensated label) are, in a postprocessing block 
108, either left and processed as individual regions or removed. 

The second circuit 92, provided for generating the order signal, comprises 
a motion field expansion block 111, an updating block 112 and a quantization block 113. The 
5 block 1 1 1 assigns for the entire image the motion field deduced from the motion parameters 
assigned to each concerned region, and the block 112 then carries out the updating of the 
motion information. The block 113 achieves a quantization of the order information by 
generating a binary number corresponding, for each pair of examined labels (i.e. of 
concerned regions), to the label corresponding to the greater number of occurrences of a 

10 situation where said label has been declared to be in the foreground of the other one. 

Obviously, other implementations of the coding system are possible, and 
especially an implementation in the form of module(s) including a processor or a 
microprocessor that ensures the performance of series of instructions corresponding to the 
functions, computations and other operations carried out within the previously described 

15 coding system. 

Moreover, the invention also relates to a coded signal as defined at the 
output of such a coding system (whatever its embodiment), and consequently to any storage 
medium on which that signal is stored (if it is not immediately transmitted or, when it is 
transmitted, if it is not immediately used at the reception side). This coded signal includes 

20 several kinds of information. It first includes a coded signal corresponding to the texture 
signal of each region of the current partition and to the corresponding motion information 
that characterizes the evolution of said region between the previous partition of the 
segmented sequence and the current one and allows to compensate its displacement from the 
former to the latter. It also includes, as they have been previously defined, the order signal 

25 defining for each considered pair of regions in conflict which of them is in the foreground of 
the other one and the corresponding updated motion information. If such a complex coded 
signal is generated, it is however not necessarily transmitted immediately, but only stored in 
order to be transmitted later, or it may be transmitted but not used immediately at the 
reception side. In both cases, this coded signal will be stored, and the invention relates to the 

30 storage medium on which this signal is stored. 

Moreover, the invention is obviously not limited to the previous 
embodiments and further relates to a method for decoding a coded signal such as described 
above, and to a corresponding decoding sub-system. The decoding method, illustrated in Fig. 
12, involves the following steps : 
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an intra decoding step 121a, a motion compensation step 122a, and a partition reconstruction 
step 123a, the output of which both constitutes the current reconstructed partition and is used 
for the following decoding step. 

The first decoding step 121a, carried out, as indicated in Fig. 13, by an 

5 intra region decoding circuit 130, is provided for a restitution of the contours of the regions 
that have been sent in intra-frame mode. The corresponding decoding circuit comprises a 
decoding stage 132 that allows to reconstruct the regions sent in intra-frame mode, according 
to the specific technique that has been used for the shape coding at the transmitting side (for 
example, in the case of a chain code method such as described for instance in the 

10 communication "Spiral image compression by an unrestricted chain coding method", P.N. 
Nicholl and RJ. Millar, 5th International Conference on Image Processing and its 
Applications, UK, 4-6 July 1995, pp.564-568, the basic method for such a reconstruction is 
to read from the input buffer 131 of the decoding circuit the location of the starting point of 
the contours and then to read the successive displacements of the chain, such as right-turn, 

15 left- turn, straight ahead, in order to draw the decoded contours). Once the set of contours is 
drawn, a mask definition stage 133 allows to assign the various labels to each region and a 
zero label to the other regions (that will be transmitted in inter-frame mode). A mask 
indicating these regions is therefore created. 

The second motion compensation step 122a can be done as previously 

20 indicated (the corresponding compensating circuit 140 will therefore not be described in a 
more detailed manner), the order information being used to solve conflicts appearing when 
two different labels may be assigned to the same location (the label corresponding to the 
foreground region as defined by said order information is assumed to be the correct one). 
The compensated partition is made of intra-frame coded regions, motion compensated regions 

25 and holes corresponding to empty areas {where no labels have been assigned during the 
compensation process). 

Once this compensated partition is available, the third partition 
reconstruction step 123a, carried out by a decoding circuit 150, allows to decode (in an error 
region decoding stage 151) the small contour segments that form the error regions. A similar 

30 process as the one used for the decoding stage 132 can be used, the main difference being 
now that one deals no longer with the whole contour of a region but only with pieces of 
contour. At the output of the decoding stage 151, a so-called over-partition, made of the 
intersection between the compensated regions and the set of transmitted error regions, is 
available. A label decoding stage 152 then allows to decode the labels of the regions of this 
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overpartition, and these labels are assigned. 

Obviously the invention also relates to a method of coding a sequence in 
which the images are already available under the form of successive segmented images 
comprising a plurality of regions and associated labels defining partitions. The method then 
5 does not comprise the first transforming step, and is consequently carried out by a coding 
system that no longer comprises a segmentation device, since successive partitions 
comprising a plurality of regions are already available. 



4 
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1. A method of coding an image sequence, comprising the steps of : 
transforming the images of said sequence into successive segmented 

images comprising a plurality of regions and associated labels defining partitions ; 

estimating for the regions of the current partition a motion information 
5 about the evolution of said segmentation from the previous partition ; 

encoding the contour and texture of each region of said current partition ; 
generating on the basis of said encoded partition a corresponding 
reconstructed partition ; 

in which the motion information estimation step comprises in cascade the sub-steps of : 
10 - defining for each region of said current partition, on the basis of a 

comparison between this current partition and a partition compensated from said 

reconstructed partition and said motion information, an intra-frame or interframe contour 

transmission mode ; 

- defining for each pair of neighbouring regions of said current partition, 
15 on the basis of a comparison between respective motion informations associated to these two 

regions, an auxiliary order information and an updated motion information. 

2. A coding method as claimed in claim 1, including a contour transmission 
mode definition sub-step in which the order information is sent backwards, in order to 
constitute a loop in which said order information updates the definition of said contour 

20 transmission mode. 

3. A coding method as claimed in claim 2, in which : 

(A) said contour transmission mode definition sub-step comprises for each 
region individually the operations of : 

compensating on the basis of the motion information associated with each 
25 successive region of the current partition said corresponding reconstructed partition ; 

computing on the basis of a comparison between said current partition and 
said corresponding compensated partition a compensation error ; 

generating as a motion information and on the basis of a comparison 
between coding costs of said current partition and said compensation error a decision related 
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to the transmission mode of the contour of the concerned region ; 

(B) said auxiliary order information definition sub-step comprises the 

operations of : 

- defining a motion vector for each position inside the concerned region 
5 and assigning said vector to all points of the current partition ; 

- generating the order information corresponding to each of the concerned 
regions, on the basis of a comparison between said current partition and said reconstructed 
partition when considering said assigned vector ; 

- quantizing said order information by comparing for each pair of labels 
10 associated with two concerned regions the number of occurrences where the first label has 

been declared to be in the foreground of the second one and the reciprocal number and 
selecting the largest one. 

4. A system for coding an image sequence, comprising : 

a segmentation device, for generating successive partitions comprising a 
15 plurality of regions ; 

a motion estimation device, for the evaluation of the motion of said 
segmentation from a previous partition to the current one ; 

an order information evaluation device, for defining a priority in the 
coding order of said regions ; 
20 - a coding device, for coding contour and texture signals for all regions of 

the current partition ; 

a partition reconstruction device ; 
in which said order information evaluation device is a decision-directed sub-system 
comprising a transmission mode definition circuit, provided for assessing the coding costs of 
25 the contour information in intra-frame and inter-frame modes and defining a first updated 
motion output signal including an updated motion information and a corresponding 
transmission mode information, and an order computation circuit, provided for associating to 
said first updated motion signal a second order signal and itself comprising : 

- a motion field expansion block, in the form of a first scanning loop for 
30 extending, for each region, its motion model to all points of the partition and defining an 

order information for said concerned region, and of a second scanning loop for iterating said 
motion field expansion for all regions of each current partition ; an updating block ; 

- a quantization block, for taking a majority rule decision related to the 
final order of each region ; the output of said order information evaluation device associating 
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said final order signal and said updated motion signal. 

5. A coded signal corresponding to each region of each successive partition 

of a segmented image sequence and comprising, for each pair of regions of a partition 
considered as in conflict as soon as their contour and texture motions differ, a first coded 

5 signal, corresponding to the texture signal of each concerned region and to the corresponding 
motion information that characterizes the evolution of said region between the previous 
partition and the current one, and auxiliary signals, corresponding to an order signal, 
defining for each considered pair of regions said in conflict which of them is in the 
foreground of the other one, and to an associated updated motion information. 

10 6. A storage medium for storing a coded signal corresponding to each region 

of each successive partition of a segmented image sequence and comprising, for each pair of 
regions of a partition considered as in conflict as soon as their contour and texture motions 
differ, a first coded signal, corresponding to the texture signal of each concerned region and 
to the corresponding motion information that characterizes the evolution of said region 

15 between the previous partition and the current one, and auxiliary signals, corresponding to an 
order signal, defining for each considered pair of regions said in conflict which of them is in 
the foreground of the other one, and to an associated updated motion information. 
7. A method of decoding coded signals corresponding to the regions of each 

successive partition of a segmented image sequence, each of said coded signals comprising 

20 for each pair of regions of a partition considered as in conflict as soon as their contour and 
texture motions differ, a first coded signal, corresponding to the texture signal of each 
concerned region and to the corresponding motion information that characterizes the 
evolution of said region between the previous partition and the current one, and auxiliary 
signals, corresponding to an order signal, defining for each considered pair of regions said in 

25 conflict which of them is in the foreground of the other one, and to an associated updated 
motion information, said decoding method comprising the steps of : 

(a) decoding the regions that have been previously coded in intra-frame 
mode and creating a corresponding mask by assigning the corresponding labels to these 
regions and a zero label to the other ones ; 

30 (b) generating a compensated partition made of intra-frame coded regions, 

motion compensated regions and holes corresponding to empty areas ; 

(c) decoding the error regions and generating a socalled over-partition 
made of the intersection between the compensated regions and the set of transmitted error 
regions, the labels of the regions of said over-partition being decoded and assigned. 
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8. A system for decoding coded signals corresponding to the regions of each 

successive partition of a segmented 27 image sequence, each of said coded signals 
comprising for each pair of regions of a partition considered as in conflict as soon as their 
contour and texture motions differ, a first coded signal, corresponding to the texture signal of 

5 each concerned region and to the corresponding motion information that characterizes the 
evolution of said region between the previous partition and the current one, and auxiliary 
signals, corresponding to an order signal, defining for each considered pair of regions said in 
conflict which of them is in the foreground of the other one, and to an associated updated 
motion information, said decoding system comprising in cascade an intra region decoding 

10 circuit, provided for decoding the regions that sent in intra-frame mode, a compensating 
circuit, provided for generating a compensated partition made of intraframe coded regions, 
motion compensated regions and holes corresponding to empty areas, an error region 
decoding stage, provided for generating a so-called over-partition made of the intersection 
between the compensated regions and the set of transmitted error regions, and a label 

15 decoding stage, provided for decoding and assigning the labels of the regions of said 
over-partition. 
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